Package recognition device, package sorting system and package recognition method

ABSTRACT

There is provided a parcel recognition device that recognizes a parcel based on an image including the parcel, the device including: a processor; and a memory, in which by cooperating with the memory, the processor recognizes a person in a recognition target space in the image, subtracts a space within a predetermined range including at least a part of the recognized person from the recognition target space, and recognizes the parcel by using a space obtained by the subtraction as a new recognition target space.

TECHNICAL FIELD

The present disclosure is related to a parcel recognition device, aparcel sorting system, and a parcel recognition method useful to sortparcels.

BACKGROUND ART

As recent economic activity rises, the amount of parcel circulationtends to increase. In a circulation process of parcels, sorting work forsorting the parcel by destination is a time-consuming process and relieson manual work from before, but a technology of automating at least apart of the sorting work is proposed.

PTL 1 discloses a system in which a moving parcel is tracked, an imageto be displayed is determined based on information related to the parcelread from the parcel and information of a position of the parcel, andthe image is projected from a projector to display the image on theparcel.

CITATION LIST Patent Literature

PTL 1: U.S. Pat. No. 7,090,134

SUMMARY OF THE INVENTION

Meanwhile, in recent years, the amount of parcel circulation has beenincreased more and more and types of parcels have also become various,so that a technology of effectively and precisely sorting the parcel isrequired.

The present disclosure is related to the technology of effectively andprecisely sorting the parcel.

According to the present disclosure, there is provided a parcelrecognition device that recognizes a parcel based on an image includingthe parcel, the device including: a processor; and a memory, in which bycooperating with the memory, the processor recognizes a person in arecognition target space in the image, subtracts a space within apredetermined range including at least a part of the recognized personfrom the recognition target space, and recognizes the parcel by using aspace obtained by the subtraction as a new recognition target space.

According to the present disclosure, there is provided a parcel sortingsystem including: a projection instruction device including the parcelrecognition device described above; a label reader that reads parcelidentification information from a label attached to a parcel; an imagesensor that obtains an image including the parcel; and an imageprojection device that projects a projection image indicating a sortingdestination on the parcel.

According to the present disclosure, there is provided a parcelrecognition method of recognizing a parcel based on an image includingthe parcel, the method including: by causing a processor to cooperatewith a memory, performing a recognition on a person in a recognitiontarget space in the image; subtracting a space within a predeterminedrange including at least a part of the recognized person from therecognition target space; and setting a space obtained by thesubtraction as a new recognition target space, and recognizing theparcel.

According to the present disclosure, it is possible to more effectivelyand precisely sort parcels and to further deal with an increase in theamount of parcel circulation. In particular, even if a person touches aparcel and interferes with the parcel, an error in position informationof the parcel and occurrence of erroneous recognition can be suppressed,and it is possible to more reliably recognize the parcel and to furthertrack the parcel with high accuracy.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a parcelsorting system according to an embodiment.

FIG. 2 is a block diagram illustrating a configuration of a projectioninstruction device according to the embodiment.

FIG. 3A is a conceptual diagram illustrating a status in which theparcel sorting system is installed in a distribution center and is inoperation.

FIG. 3B is a diagram illustrating a state in which a projection imageincluding a number is projected on an upper surface of a parcel.

FIG. 4 is a diagram illustrating an example of a projection imagegenerated by the projection instruction device according to theembodiment.

FIG. 5 is a flowchart illustrating an outline procedure of an operationmainly performed by the projection instruction device.

FIG. 6 is a diagram for explaining a recognition process for a parcel ina case where a person interferes with the parcel.

FIG. 7 is a flowchart illustrating the recognition process for a parcelin a case where a person interferes with the parcel.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments (hereinafter, referred to as “presentembodiment”) which specifically disclose a parcel recognition device, aparcel sorting system, and a parcel recognition method according to thepresent disclosure will be described in detail with reference toappropriate drawings. Meanwhile, in some cases, an unnecessarilydetailed explanation may be omitted. For example, in some cases, adetailed explanation of already well-known items and a repetitionexplanation of substantially the same configuration may be omitted. Thisis to avoid unnecessary repetition of the following description and tofacilitate understanding by those skilled in the art. The accompanyingdrawings and the following description are provided to enable thoseskilled in the art to fully understand the present disclosure and arenot intended to limit a scope of the claims.

Hereinafter, the embodiments of the disclosure will be described withreference to FIGS. 1 to 7.

[Configuration]

FIG. 1 is a block diagram illustrating a configuration of the parcelsorting system according to the embodiment. Parcel sorting system 100illustrated in FIG. 1 is installed in, for example, a distributioncenter. Parcel sorting system 100 includes label reader 10, image sensor20, projection instruction device 30, and projector 40. Parcel sortingsystem 100 is a system which supports work of a worker who sorts parcelstransported by a transport conveyor. Parcel sorting system 100 isinstalled at the distribution center owned by, for example, a retailer,a wholesaler, an internet distributor, or the like. A parcel to besorted generally has an approximately rectangular parallelepiped shape,but the outward shape of the parcel is not particularly limited, and atype of the parcel is also not particularly limited. The configurationof the parcel sorting system is not limited to the configurationillustrated in FIG. 1. For example, one label reader 10 may be connectedwith a plurality of image sensors 20, a plurality of projectioninstruction devices 30, and a plurality of projectors 40. The number ofeach component can be appropriately modified according to the purpose.

Label reader 10 as a reading device is a device which includes variouscomponents such as a lens (not illustrated), an image sensor, and thelike. By using label reader 10, it is possible to read label recordinginformation recording various types of information related to the parcelfrom a label attached to the parcel transported by the transportconveyor. By using the read label recording information, it becomespossible to specify the parcel. By the read information, parcelidentification information is defined.

Image sensor 20 is an imaging device which includes various componentssuch as a lens (not illustrated), an image sensor, and the like. Imagesensor 20 is generally configured by an imaging camera. The imagingcamera is a three-dimensional camera, a plurality of two-dimensionalcameras, or the like. Image sensor 20 includes distance image sensor 22and color image sensor 24.

Distance image sensor 22 images the parcel transported by the transportconveyor and generates a distance image. The generated distance image isused as information indicating a position of the parcel, a distance tothe parcel, a size of the parcel, and the like. “Distance image” meansan image including distance information indicating a distance from animaging position to a position (including a surface of a parcel)indicated by each of pixels (that is, “image” in the present disclosureincludes a distance image). In addition, a term of “distance image”includes one which cannot be recognized as an image by human eyes, suchas a table or the like listing numerical values indicating a distance.That is, “distance image” may be information indicating a relationshipbetween coordinates and a distance in the imaged region, and a datastructure is not limited thereto. In the present disclosure, distanceimage sensor 22 is used for specifying the position of the parcel.Therefore, distance image sensor 22 also can be replaced with anothersensing device (an ultrasonic sensor, an infrared sensor, a stereocamera, and a monocular video camera).

Color image sensor 24 images a parcel generated by the distance imageand generates a color image. “Color image” refers to an image in which acolor of a surface of a parcel is expressed with a predeterminedgradation, and the “gradation” includes not only 256 gradations of RGBbut also all kinds of grayscales or the like. Color image sensor 24 inthe present disclosure is used for tracking each parcel, for the parcelspecified by distance image sensor 22. Color image sensor 24 also can bereplaced with another sensing device (an ultrasonic sensor, an infraredsensor, a stereo camera, and a monocular video camera).

That is, in the present disclosure, a term of “image” includes both of adistance image and a color image. In the present disclosure, informationoutput from an image sensor as a sensing device including a distanceimage sensor and a color image sensor is referred to as sensinginformation. In the present embodiment, an example of the sensing devicewill be described by using image sensor 20 (including distance imagesensor 22 and color image sensor 24). In addition, in the presentembodiment, an example of the sensing information will be described byusing a distance image output by distance image sensor 22 and a colorimage output by color image sensor 24.

Projection instruction device 30 functions as a calculation device inparcel sorting system 100. As illustrated in FIG. 2, projectioninstruction device 30 includes input unit 32, processor 34, memory 36,and output unit 38 connected with one another via a bus. Input unit 32receives parcel identification information for specifying a parcelobtained from the label recording information read by label reader 10, adistance image generated by distance image sensor 22, and a color imagegenerated by color image sensor 24. Processor 34 is configured by ageneral calculation device and generates a projection image to beprojected on a parcel based on the parcel identification information,the distance image, and the color image. Memory 36 as a storage devicereads a control program necessary for various processes by processor 34and performs an operation such as data backup. That is, processor 34 andmemory 36 control various processes by projection instruction device 30by cooperating with each other. Output unit 38 outputs the projectionimage generated by processor 34 to projector 40. In the presentdisclosure, “processor” does not mean only a single processor.“Processor” means an operator having a plurality of processors having anidentical purpose, or a plurality of processors having differentpurposes in a case where they perform processes in collaboration withone another (for example, general-purpose central processing unit (CPU)and a graphic processing unit (GPU)).

Projector 40 is configured by a general projection device, and projectsprojection light including the projection image received from projectioninstruction device 30, on the parcel and displays the projection imageonto the parcel.

Parcel sorting system 100 can be configured to include label reader 10,image sensor 20 (distance image sensor 22 and color image sensor 24),projection instruction device 30, and projector 40 connected with oneanother in wired communication or in wireless communication. Inaddition, parcel sorting system 100 also can be configured to includetwo or more devices of any of label reader 10, image sensor 20,projection instruction device 30, and projector 40 as an integraldevice. For example, image sensor 20 and projector 40 can be combined toconstruct an integral imaging projection device (see FIG. 3A).

[Outline of System]

FIG. 3A is a conceptual diagram illustrating a status in which parcelsorting system 100 is installed in the distribution center and is inoperation. In the related art, each of workers M visually checks thelabel attached to each parcel P transported by transport conveyor 50 inthe arrow X-direction. When the parcel to be delivered by the workerhimself arrives, worker M needs to pick up the parcel and places theparcel once in the vicinity such as the worker's own feet, a basket, atruck bed. Meanwhile, in a case of visually sorting by the worker, thereis a limit in work efficiency of the worker, so that it is necessary tolimit a transport speed to a predetermined value or less. As a result, alimit value of the number of parcels which the worker can sort in a unittime, was few. In addition, due to an erroneous recognition when theworker visually checks the label, there is a possibility that an errormay occur at the time of sorting. In recent years, the amount of parcelcirculation has increased, and these problems receive more attention.

In the present embodiment, as illustrated in FIG. 3A, label reader 10disposed above transport conveyor 50 reads the label attached to eachparcel P transported by transport conveyor 50. In the label, the labelrecording information including various information related to theparcel is described. The label recording information includesinformation similar to a parcel identification number individuallyassigned to the parcel, a name, an address, and a telephone number of asender, a name, an address, and a telephone number of a receiver, aparcel type, and the like. The label may be read by the worker in chargeby manually placing a barcode reader as label reader 10 to a barcode inthe label.

Further, image sensor 20 images the image (the distance image and thecolor image) of parcel P transported by transport conveyor 50 andobtains information such as a position of parcel P, a distance to parcelP, a size (lengths of three sides when parcel P is rectangularparallelepiped) of parcel P, a color of parcel P, a pattern of parcel P,and the like. Further, positions of label reader 10 and image sensor 20,a type of the sensing device, and an order of processes are notparticularly limited to the illustrated embodiments. As described above,in the present example, image sensor 20 and projector 40 are configuredas integrated imaging projection device 60 and are disposed abovetransport conveyor 50.

Projection instruction device 30 (not illustrated in FIGS. 3A and 3B) isconfigured by a computer disposed in, for example, the vicinity oftransport conveyor 50 or another room and generates the projection imageto be displayed on parcel P (for example, upper surface when parcel P isrectangular parallelepiped) based on the information, obtained by labelreader 10, specifying the parcel and the distance image and the colorimage generated by image sensor 20. Projection instruction device 30transmits a projection instruction to project the projection image onparcel P, to projector 40.

Projector 40 which receives the projection instruction, as an imageprojection device, projects projection light including the projectionimage generated by projection instruction device 30 on parcel P anddisplays the projection image on parcel P. Here, the projection imagedisplayed on parcel P is, for example, an image of an encircled numberhaving a color indicating a sorting location corresponding to a deliveryaddress of parcel P (see FIG. 3B). Here, the encircled numbercorresponds to, for example, a number of a truck carrying sorted parcelP (a number of the truck itself, a parking lot number, or the like), anumber of a shelf or a box to be carried into the truck, or the like.Further, instead of directly corresponding to the number such as theshelf or the box, the encircled number may correspond to a number of ashooter which moves the picked-up parcel to another location or a truck.Since a parking position of the truck or the like frequently changesaccording to a traffic condition or the like, it may be difficult tocorrespond to the sorting destination viewed from the periphery oftransport conveyor 50 at any time. Therefore, the shooter is sandwichedbetween transport conveyor 50 and the transport truck, and a number ofthe shooter is projected on the periphery of transport conveyor 50, sothat even if a configuration on the periphery of the transport conveyor50 is not changed as needed, it is possible to deal with the change ofthe sorting destination by disposing an opening of the shooter.Certainly, according to a status, various types of projection images aredisplayed. Another example of displaying the number may be a zip codecorresponding to the delivery address, a number of a worker who picks upparcel P, or the like. In addition, as an example of displayinginformation other than the number, an arrow indicating the sortingdirection (such as right or left in a transport direction of transportconveyor 50) or characters (such as “left” and “right”) may be used.Further, a display form is not limited to the encircled number, andvarious types such as numbers surrounded by squares (“3”, “359”, and“24735”) as illustrated in FIG. 4 are conceivable. Furthermore, theprojection image is not limited to numbers or characters enclosed with aframe, but may be white numbers or characters with a solid background.In addition, the shape of the number or character to be displayed suchas a circle, a triangle, a square, or the like may be switched accordingto information to be displayed. Further, a picture capable of beingindividually associated with each information to be displayed may bedisplayed. The projection image is not limited to a still image, and maybe an animation. As an example of the animation, the example describedabove may be blinked, enlarged or reduced, and a color may be changed.An animation reflecting the sorting direction may be projected. Anexample of the animations reflecting the sorting direction may includevarious patterns, for example, a moving light ray or light spot in thesorting direction, forming all or a part of the projection image in thesorting direction, changing the color, moving and displaying an arrow inthe sorting direction, and the like. In a case where only a part of theprojection image is a target of the animation, a part having a largeinfluence on determination of the sorting destination by a worker, suchas a number, an arrow, or the like, may be not changed, and a part notaffected by the sorting destination such as a frame line may be changed.Meanwhile, in a situation in which it is more efficient to convey thesorting direction more intuitively than the meaning of the number or thelike projected within the frame line, such as a case where there are fewoptions for the sorting destination, a number, an arrow, or the like maybe moved in the sorting direction within a fixed frame line. Theanimation may be repeatedly projected or may be projected only once. Inthe present disclosure, “image projection device” is not limited to adevice which directly projects a light beam on the parcel. In thepresent disclosure, “image projection device” includes glasses which candisplay the image. That is, in the present disclosure, in a case ofexpressing as projecting projection light on the parcel, displaying theimage on the parcel, projecting the image on the parcel, or the like,the expression also includes allowing the worker to recognize the imagevia the glasses capable of displaying the image in a pseudo manner as ifthe projection light is projected on the parcel. That is, in a casewhere the worker wears special glasses capable of displaying the image,the projection image of parcel P may be superimposed on the image ofparcel P viewed via the glasses.

In FIG. 3A, worker M (omitted for other workers) in charge of picking upthe parcel stands beside transport conveyor 50 and the parcel arrivingeach region is picked up from transport conveyor 50 as indicated byencircled number 1, encircled number 2, encircled number 3, or the like.

For example, parcel P1 has parcel identification information of “AAA111”on a label, and the parcel identification information of “AAA111”specifies that the parcel is a target to be sorted in region A. Here,when parcel P1 reaches the specific region, processor 34 transmits thegenerated projection image to projector 40 as illustrated in FIG. 3B.Projector 40 projects the projection image on parcel P1. The worker inthe region can easily pay attention to parcel P1, to be picked up by theworker, which reaches the specific region of the worker, accordingly,the parcel can be sorted more efficiently and precisely.

In the present embodiment, as illustrated in FIG. 3A, whether or not oneprojector 40 projects the image on a plurality of specific regions maybe switched and whether or not a plurality of projectors 40 project theimage on each of the specific regions may be switched.

Hereinafter, in parcel sorting system 100 according to the embodiment,an outline of an operation of sorting the parcel performed by projectioninstruction device 30 will be described.

[Outline of Operation]

FIG. 5 is a flowchart illustrating an outline procedure of an operationof mainly performed by projection instruction device 30 of the presentembodiment, particularly processor 34 of projection instruction device30. First, after label reader 10 reads the label recording informationof the label of the parcel, input unit 32 of projection instructiondevice 30 obtains the parcel identification information corresponding tothe label recording information from label reader 10 (step S1). Theparcel identification information is information including at least onepiece of information similar to the parcel identification numberindividually assigned to the parcel, the name, the address, and thetelephone number of the sender, the name, the address, and the telephonenumber of the receiver, the parcel type, and the like. For the parcelidentification information, processor 34 assigns an ID as a parcelidentification number specifying the parcel and records the ID and timeinformation corresponding to a time at which the ID is assigned, inmemory 36 (step S2). The ID recorded in memory 36 may be the parcelidentification number originally recorded in the parcel identificationinformation or may be assigned by projection instruction device 30generating a new ID.

On the other hand, in parallel with step S1 and step S2, after distanceimage sensor 22 of image sensor 20 images the distance image of theparcel, input unit 32 of projection instruction device 30 obtains thedistance image as the sensing information from distance image sensor 22(step S20). Processor 34 determines whether or not the ID correspondingto the parcel existing in the distance image exists in memory 36.

An example of a method of determining whether or not the IDcorresponding to the parcel existing in the distance image exists inmemory 36 is as follows. That is, processor 34 calculates a timerequired for the parcel to move between label reader 10 and distanceimage sensor 22 by a distance (assumed to be known) between label reader10 and distance image sensor 22 and a speed of transport conveyor 50. Bysubtracting the time from a time when the distance image is obtained, atime when the ID is assigned to the parcel existing in the distanceimage by label reader 10 (or processor 34) can be estimated. It can beestimated that the ID assigned close to the estimated time is the IDcorresponding to the parcel existing in the distance image. In addition,as another example, a method of installing another distance image sensorin the vicinity of label reader 10 may be possible. That is, by trackingthe parcel, to which the ID is assigned, by using another distance imagesensor installed in the vicinity of label reader 10 since label reader10 assigns the ID (or processor 34), a distance between the parcel (orthe ID) and label reader 10 is measured for a time unit. Processor 34can estimate the ID of the parcel in the distance image obtained inpredetermined step S20 by the measured distance between the parcel (orthe ID) and label reader 10, a distance of the parcel in the distanceimage obtained in step S20, and a distance (assumed to be known) betweentwo distance image sensors.

In this manner, processor 34 determines whether or not the IDcorresponding to the parcel included in the distance image exists inmemory 36 (step S30). That is, as described in step S2, in memory 36,the parcel identification information, the ID, and the time informationcorresponding to a time when the ID is assigned are recorded in advance.On the other hand, as described above, for example, processor 34subtracts the time required for the parcel to move between label reader10 and distance image sensor 22 from the time when the distance image isobtained, a time when the ID is assigned to the parcel existing in thedistance image by label reader 10 (or processor 34) can be estimated.Processor 34 compares the time information recorded in memory 36 inadvance and the estimated time. In a case where the time information isclose to the estimated time (for example, a case where a time differenceis equal to or smaller than a predetermined threshold time), processor34 can determine that the ID corresponding to the parcel included in thedistance image exists in memory 36. In a case where it is determinedthat the ID corresponding to the parcel exists in memory 36 (Yes in stepS30), the process moves to step S60 and subsequent steps.

In a case where it is determined that the ID corresponding to the parceldoes not exist in memory 36 (No in step S30), on the premise that the IDis not assigned to the parcel, processor 34 specifies the position ofthe parcel again (step S40) and assigns the ID to the parcel (step S50).

In parallel with the above steps, color image sensor 24 generates acolor image for each parcel of which a distance image is obtained.Processor 34 tracks the parcel, to which the ID is attached, transportedand moved by transport conveyor 50 based on the color image from colorimage sensor 24 obtained by input unit 32 (step S60). Based on the colorimage likewise, processor 34 determines whether or not the worker picksup the tracked parcel (step S70). In a case where it is determined thatthe parcel is not picked up by the worker (No in step S70), processor 34determines whether or not the parcel exists in a specific region (apredetermined sorting area in which the parcel is to be picked up) to bedescribed below. In a case where it is determined that the parcel exists(reaches) in the specific region (Yes in step S80), processor 34generates the projection image and transmits the projection image toprojector 40 (step S90). In a case where it is not determined that theparcel exists (reaches) in the specific region (No in step S80), theprocess returns to step S60 and processor 34 continues to track theparcel.

In addition, in step S70, in a case where it is determined that theparcel is picked up by the worker (Yes in step S70), processor 34 readsdetail information of the parcel from memory 36 (step S100), generatesthe projection image including the detail information, and outputs theprojection image generated by output unit 38, to projector 40 (stepS90). Projector 40 which obtains the projection image from projectioninstruction device 30, projects the projection image on thecorresponding parcel.

The above is the outline of the operation procedure performed byprocessor 34 and the like of projection instruction device 30.Certainly, the operation procedure is not limited to that describedabove. For example, the determination in step S70 can be omitted. Inaddition, for the determination in step S70, it is possible to usecontact determination between a hand of the worker and the parcel, acolor image, a distance image, or the like. Hereinafter, furtherdetailed procedures of each process will be described.

[Process in Case where Person Interferes with Parcel]

In a case where a parcel is recognized based on a distance imageincluding the parcel, if a person touches the parcel and interferes withthe parcel, an error may occur in position information or an erroneousrecognition may occur. For example, when a person is in contact with aparcel, the arm of the person is also recognized as a part of theparcel, and a shape or the like of the parcel may be erroneouslyrecognized. In such a case, there is a possibility that the parcelcannot be accurately tracked.

In the present disclosure, in step S60 in FIG. 5, a person is recognizedin a recognition target space inside a distance image, a space within apredetermined range around the recognized person is subtracted from therecognition target space, and a space obtained by the subtraction is setas a new recognition target space so that the parcel is reliablyrecognized.

That is, in a recognition target space inside a distance image includinga parcel which is an output of distance image sensor 22, as sensinginformation, processor 34 recognizes a person by using an existing imagerecognition technology, then, subtracts a space within a predeterminedrange around the recognized person from the recognition target space,and sets a space obtained by the subtraction as a new recognition targetspace, so that the parcel is recognized in the new recognition targetspace.

FIG. 6 is a diagram for explaining a recognition process for a parcel ina case where a person interferes with the parcel. In the drawing, in acase where there is one transport conveyor BCa on the upstream side andthree transport conveyors BCb to BCd on the downstream side, person Wk(worker) disposed between transport conveyor BCa and three transportconveyors BCb to BCd picks up parcel P transported by transport conveyorBCa on the upstream side, and performs sorting work of placing parcel Pon one of transport conveyor BCb, transport conveyor BCc, and transportconveyor BCd. In such sorting work, when person Wk touches parcel Ptransported by transport conveyor BCa and interferes with parcel P, anerror may occur in parcel information or an erroneous recognition mayoccur. In such a case, person Wk is first recognized, then predeterminedrange Rs around person Wk is determined, and a space within determinedpredetermined range Rs is subtracted from the recognition target space.A space obtained in this manner is set as a new recognition targetspace. In this manner, even if person Wk touches parcel P and interfereswith parcel P, an error in position information of parcel P oroccurrence of erroneous recognition can be suppressed, and it ispossible to more reliably recognize parcel P and to further track theparcel with high accuracy. Various methods can be used for recognizingperson Wk. For example, if it is enough to roughly recognize a positionat which person Wk exists, the position of person Wk may be recognizedby a tag or a beacon attached to clothes of person Wk, a uniform havinga specific color or a shape, and the like being recognized. On the otherhand, if it is necessary to more accurately recognize the position ofperson Wk, another algorithm such as pattern matching may be used.Further, both methods may be used in combination. In order to determinerange Rs accurately, it is desirable to accurately recognize theposition of person Wk. Meanwhile, an algorithm for accuratelyrecognizing the position of person Wk generally has lower accuracy insituations in which person Wk is at a distant position or when aplurality of persons Wk densely exist. Here, by using both methodstogether, it is possible to realize a structure for recognizing personWk existing in the recognition target section as completely as possibleand with high accuracy.

FIG. 7 is an example of a flow corresponding to the process describedabove. The flow illustrated here indicates a parcel recognition processwhich is a sub-process (a subroutine) of the process in step S60 in theflowchart of the outline procedure in FIG. 5. That is, processor 34first obtains a distance image including a parcel from distance imagesensor 22 (step S401). Next, processor 34 recognizes person Wk by usingan existing image recognition technology, in a recognition target spacein the obtained distance image (step S402). After recognizing person Wk,processor 34 subtracts a space within predetermined range Rs aroundrecognized person Wk from the recognition target space (step S403). Aspace obtained by the subtraction is set as a new recognition targetspace, and parcel P is recognized (step S404). After recognizing parcelP by performing the above sub-process, processor 34 performs a processof tracking parcel P.

As described above, according to the present disclosure, even if personWk touches parcel P transported by a transport conveyor and interfereswith parcel P, an error in position information of parcel P oroccurrence of erroneous recognition can be suppressed, and it ispossible to more reliably recognize parcel P and to further track theparcel with high accuracy.

Instead of using a distance image, it is also possible to use a colorimage to recognize a person and a space within a predetermined rangearound the person.

Although the embodiment of a parcel recognition device, a parcel sortingsystem, and a parcel recognition method according to the presentdisclosure is described with reference to the drawings, the presentdisclosure is not limited to such an example. Those skilled in the artcan conceive various modification examples, change examples,substitution examples, addition examples, deletion examples, andequivalent examples within the scope described in the claims and theserightly belong to the technical scope of the present disclosure.

For example, the following modification example can be considered.

In the present disclosure, the situation is assumed in which person Wkexists at a center portion of a plurality of transport conveyors asillustrated in FIG. 6, but the present disclosure is not limitedthereto. The present disclosure can be applied to a case where person Wkexists near one transport conveyor as illustrated in FIG. 3. Even in thesituation as illustrated in FIG. 3, a situation occurs in which a personwho tries to pick up a parcel is erroneously recognized as a parcel, sothat the technology of the present disclosure is useful.

In the present disclosure, person Wk is excluded from the recognitiontarget space of the distance image, but the present disclosure is notlimited thereto. For example, person Wk may be excluded from arecognition target space of a color image. In a case where a color ofclothes of the person is similar to a color of the parcel, not only in arecognition result of the distance image but also in a recognitionresult of the color image, there is a possibility that a shape of theparcel and a shape of the person are erroneously recognized as beingintegrated. In the present disclosure, since the parcel is tracked basedon the color image, if such erroneous recognition occurs, a trackingerror of the parcel occurs. By excluding person Wk from the color image,such a tracking error can be suppressed.

In the present disclosure, as illustrated in FIG. 6, a columnar shapesurrounding person Wk is used as range Rs to be excluded from therecognition target space, but the range is not limited thereto. Ifperson Wk is excluded from the recognition target space, a size or ashape of range Rs may be different from those in FIG. 6. In addition,since it is only for preventing erroneous recognition of the parcel andperson Wk, it is not necessary to exclude whole person Wk, and forexample, only the arm in contact with the parcel may be excluded.Further, the size and the shape of range Rs may be switched according toat least one of the number of persons Wk and the number of parcelsassumed to exist in the recognition target space. That is, in a casewhere the number of persons Wk is small or the number of parcels issmall, it is more beneficial to set wider range Rs to surely excludeperson Wk from the recognition target space. On the other hand, in acase where the number of persons Wk is large or the number of parcels islarge, excluded range Rs may overlap with a position at which the parcelexists, and tracking of the parcel itself may fail. Therefore, in a casewhere the number of persons Wk is large or the number of parcel islarge, it is useful to set range Rs as close as possible to a shape ofperson Wk itself or as a narrower range. The number of parcels existingin the recognition space can be estimated from the number of parcelssuccessfully recognized at the immediately preceding timing.

In the present disclosure, recognized person Wk is used for the purposeof excluding recognized person Wk from the recognition target space, butthe disclosure is not limited thereto. For example, in the configurationas illustrated in FIG. 6, recognized person Wk may be used for a purposeor the like of determining whether or not a parcel picked up by personWk is moved to an appropriate transport conveyor. In the presentdisclosure, since range Rs including person Wk can be recognized, theparcel picked up by person Wk can be specified by specifying the parceloverlapping with range Rs. In addition, if tracking of the parcel issuccessful, it is possible to determine which transport conveyor is acorrect destination for each parcel. Therefore, by using these methods,it is possible to determine whether or not person Wk moves the picked-upparcel to an appropriate transport conveyor. In this case, if the parcelis moved to an erroneous transport conveyor, a parcel sorting system maynotify recognized person Wk of a warning by means such as a voice, aprojection image, or the like. Although the example in which adestination of the parcel is the transport conveyor is described withreference to FIG. 6, this method can also be applied to an example inwhich the parcel is moved to another destination such as a shooter or atransport truck.

INDUSTRIAL APPLICABILITY

The present disclosure is useful to provide a parcel recognition device,a parcel sorting system, and a parcel recognition method capable ofreliably recognizing a parcel even if a person touches the parceltransported by a transport conveyor and interferes with the parcel.

REFERENCE MARKS IN THE DRAWINGS

-   -   10 LABEL READER    -   20 IMAGE SENSOR    -   22 DISTANCE IMAGE SENSOR    -   24 COLOR IMAGE SENSOR    -   30 PROJECTION INSTRUCTION DEVICE    -   32 INPUT UNIT    -   34 PROCESSOR    -   36 MEMORY    -   38 OUTPUT UNIT    -   40 PROJECTOR    -   50 TRANSPORT CONVEYOR    -   60 IMAGING PROJECTION DEVICE    -   100 PARCEL SORTING SYSTEM    -   P PARCEL

1. A parcel recognition device that recognizes a parcel based on an image including the parcel, the device comprising: a processor; and a memory, wherein by cooperating with the memory, the processor recognizes a person in a recognition target space in the image, subtracts a space within a predetermined range including at least a part of the recognized person from the recognition target space, and recognizes the parcel by using a space obtained by the subtraction as a new recognition target space.
 2. The parcel recognition device of claim 1, wherein the processor subtracts a space including the whole recognized person from the recognition target space.
 3. The parcel recognition device of claim 1, wherein the processor changes at least one of a size or a shape of a range to be subtracted from the recognition target space corresponding to each person, according to the number of the recognized persons.
 4. The parcel recognition device of claim 1, wherein the processor estimates the number of parcels existing in the recognition target space and changes at least one of a size or a shape of a range to be subtracted from the recognition target space, according to the number of the estimated parcels.
 5. The parcel recognition device of claim 1, wherein the processor specifies a correct sorting destination of the parcel in a case where a range to be subtracted from the recognition target space and the recognized parcel overlap with each other, and performs a warning when the parcel is not sorted into the correct sorting destination.
 6. The parcel recognition device of claim 1, wherein the image is a distance image.
 7. The parcel recognition device of claim 1, wherein the image is a color image.
 8. A parcel sorting system comprising: a projection instruction device including the parcel recognition device according to claim 1; a label reader that reads parcel identification information from a label attached to a parcel; an image sensor that obtains an image including the parcel; and an image projection device that projects a projection image indicating a sorting destination on the parcel.
 9. A parcel recognition method of recognizing a parcel based on an image including the parcel, the method comprising: by causing a processor to cooperate with a memory, recognizing a person in a recognition target space in the image; subtracting a space within a predetermined range including at least a part of the recognized person from the recognition target space; and recognizing the parcel by using a space obtained by the subtraction as a new recognition target space. 